We propose a general modeling and inference framework that combines the complementary strengths of probabilistic graphical models and deep learning methods.

We propose a general modeling and inference framework that combines the complementary strengths of probabilistic graphical models and deep learning methods. For inference, we use recognition networks to produce local evidence potentials, then combine them with the model distribution using efficient message-passing algorithms. All components are trained simultaneously with a single stochastic variational inference objective. We illustrate this framework by automatically segmenting and categorizing mouse behavior from raw depth video, and demonstrate several other example models.

Combining PGMs and deep learning/neural networks is a very active and promising area of research. This is an interesting paper, with it's main contribution to the existing literature being that it presents a model that can account for discrete latent variables. This new capability suggests that it could be used in a variety of interesting applications, including the type of behavior representation modeling shown in one of the experiments. Overall, the organization of the paper is excellent, and the writing is clear. The technical part is however quite dense.

We propose a general modeling and inference framework that combines the complementary strengths of probabilistic graphical models and deep learning methods. For inference, we use recognition networks to produce local evidence potentials, then combine them with the model distribution using efficient message-passing algorithms. All components are trained simultaneously with a single stochastic variational inference objective. We illustrate this framework by automatically segmenting and categorizing mouse behavior from raw depth video, and demonstrate several other example models. Papers published at the Neural Information Processing Systems Conference.